Three-dimensional monitoring apparatus

ABSTRACT

To provide a three-dimensional monitoring apparatus capable of detecting with high accuracy an incoming of an object into a predetermined three-dimensional space by using pattern light as a detecting carrier. A three-dimensional monitoring apparatus comprising (1) an irradiating device for irradiating predetermined pattern light to three-dimensional space S to be monitored, (2) an imaging device for imaging a projection pattern projected by irradiating of the pattern light on an incoming object M and on a screen  5  in the space S to capture image data and (3) a measuring device to measure a position of the object M based on the comparison between a monitoring image captured by the imaging device when there is the object M in the space S and a reference image captured by the imaging device when there is no object M in the space S.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to a three-dimensional monitoringapparatus suitable for use of detecting a human body coming into, forexample, a dangerous area in a production line or an industrial machineand the like in a factory, more specifically, to a three-dimensionalmonitoring apparatus capable of acquiring position information of anincoming object into a predetermined three-dimensional space to bemonitored by using pattern light as a carrier.

[0003] 2. Description of the Background Art

[0004] In recent years, cases bringing a monitoring apparatus fordetecting the presence or absence of an incoming object in apredetermined area (a three-dimensional space to be monitored) into afactory and the like have been increased in number for the purposes ofpreventing operators from being suffered from inclusion accidents due toa variety of equipment and preventing equipments from being damaged dueto a sudden incoming of the object.

[0005] As for a monitoring apparatus, an area sensor monitoring apredetermined plane by using continuance of a plurality of light beamsand a laser-scanning-type sensor monitoring a predetermined plane withscanning laser beams by using a polygon mirror and the like are wellknown. With using these apparatuses, for example, when the incomingobject is detected, it also becomes possible to perform automaticcontrol such as 97 immediately stop of the equipments.

[0006] However, these conventional sensors (monitoring apparatuses) havebeen predicated on so called ‘two-dimensional plane monitoring’developing a virtual light detecting surface on a space to be monitoredby illuminating and generating an output by reacting only objectscrossing (blocking) the detecting surface, so that we would have to saythat these apparatuses have not been suitable for monitoring of anarbitrary three-dimensional space to be monitored. That is to say, evenin monitoring the arbitrary three-dimensional space to be monitored,there are problems such that only the presence of the incoming of anobject from a side in which the light detecting surface has beendeveloped can be monitored (detected) actually and it is impossible toreact to an object and the like incoming into the predetermined space tobe monitored without passing through the light detecting surface to bemonitored.

[0007] Furthermore, the conventional sensors basically detect only thepresence or the absence of an object blocking the light detectingsurface and it is quite impossible to match user's requests even fordetecting a state of incoming of the object into the predetermined spaceto be monitored (for example, how far the object has entered into thepredetermined space to be monitored).

SUMMARY OF THE INVENTION

[0008] The present invention is noted to the problems above describedand it is an object of the invention to provide a three-dimensionalmonitoring apparatus capable of detecting with high accuracy an incomingof the object into the predetermined space to be monitored by the usingpattern light as the detecting medium.

[0009] It is another object of the present invention to provide athree-dimensional monitoring apparatus capable of acquiring positioninformation on the incoming object into the predetermined space to bemonitored by using the pattern light as the detecting medium.

[0010] Other and further objects and effects of the present inventionwill become apparent to those skilled in the art from a reference to thefollowing specification.

[0011] The three-dimensional monitoring apparatus of the presentinvention comprises an irradiating means for irradiating predeterminedpattern light to a three-dimensional space to be monitored, an imagingmeans for imaging a projection pattern projected by irradiating thepattern light on a surface of an object existing in the space to bemonitored and on a surface of a predetermined body composing of abackground of the space to be monitored to capture image data and aposition measuring means for acquiring position information on anincoming object into the space to be monitored based on the comparisonbetween the image data captured by the imaging means when there is anincoming object in the space to be monitored and standard image datacorresponding to the image data captured by the imaging means when thereis no incoming object in the space to be monitored.

[0012] As for ‘pattern light’ above described, the ‘pattern light’means, for example, a light beam projecting an image (a certainprojection pattern) in a certain shape or a pattern when the patternlight is irradiated on a flat screen. In the description of‘projecting’, the foregoing ‘pattern light’ includes such light with awavelength which is not recognized by human eyes as an infrared ray andthe like, and as an example for ‘a certain shape or a pattern’, apattern like a grid, a pattern like concentric circles, a pattern likeconcentric polygons and the like are concerned.

[0013] As a specific example for ‘an irradiating means’ a laserirradiating apparatus is concerned and as an example for ‘an imagingmeans’, a CCD (charge coupled device) camera is concerned.

[0014] In the description of ‘a surface of an object existing in thespace to be monitored and on a surface of a body composing of abackground of the space to be monitored’, a prepared flat screen body ispreferably used as the applicable object, so that reflection-efficiencyof the pattern light is improved and measurement stability is ensured.In the use of the flat screen body, the ‘a body composing of abackground’ above described is needed to be prepared separately, but itis possible for such a ‘wall’ which has been existing there since thestart of the space to be concerned as the body.

[0015] Moreover, as the description of ‘corresponding to image data’, itis possible for the standard image data not to be acquired directlythrough the imaging means, in other words, it is possible for thestandard image to be inputted directly through an external device suchas a PLC (programmable logic controller) so long as the standard imageis the image which is ‘corresponding to the image data captured by theimaging means when there is no incoming object in the space to bemonitored’.

[0016] The three-dimensional monitoring apparatus of the presentinvention monitors by using the pattern light as the carrier (detectingmedium), so that the monitoring apparatus can perform not only aflat-type (one-dimensional) monitoring but also a monitoring of theincoming object into the three-dimensional space, and the monitoringapparatus measures an existence position of the object, so that theapparatus is suitable not only for use of detection for existing of theincoming object but also for use of monitoring in the various types ofthree-dimensional space and use of position measurement, etc.

[0017] In the monitoring apparatus of the present invention, preferably,a decision-output means for outputting a device control signal based onthe position information calculated by the position measuring means isfurther equipped.

[0018] According to the manner as stated above, it is possible to stopurgently a predetermined facility or device based on the incoming of theobject into the space to be monitored and construct easily such a safetysystem as an automatic operation of an warning lamp, so that it ispossible to prevent operators from being suffered from inclusionaccidents caused by each machine and facility

[0019] In the monitoring apparatus of the present invention, preferably,the standard image data is acquired by an instruction by imaging throughthe imaging means.

[0020] According to the manner as stated above, it is possible to stopurgently a predetermined facility or device based on the incoming of theobject into the space to be monitored and to construct easily such asafety system as an automatic operation of an alert lamp, so that it ispossible to prevent operators from being suffered from inclusionaccidents caused by each machine and facility.

[0021] Furthermore, generation or occurrence of the pattern light can beconcerned in a variety of manners, as the most preferable example, theirradiating means of the monitoring apparatus is composed of such alight source as a laser diode oscillating laser beam and a pattern lightgenerator converting the laser beam irradiated from the light sourceinto the predetermined pattern light.

[0022] As for the expression of ‘a pattern light generator’, an MLA(micro lens alley) or a CGH (computer generated holograph) is suitablefor the generator. The MLA and the CGH are generally having highdurability because of having no movable parts in them and possible to bereduced in weight of the bodies themselves.

[0023] In the monitoring apparatus of the present invention, preferably,the irradiating means has a scanning mechanism capable of scanning thepattern light in the predetermined area by controlling the direction ofirradiation of the pattern light and the imaging means is composed tocapture the image data corresponding to a combined projection of aplurality of instant projection patterns projected to the space to bemonitored in a predetermined direction and at a predetermined timingthrough the scanning of the pattern light in the space to be monitoredby means of the scanning mechanism.

[0024] In the manner stated above, it becomes possible to change andadjust range to be monitored based on the adjustment of a scanning rangeand also to change and adjust a detecting resolution based on theadjustment of the combined projection.

[0025] As for change and adjustment described above, the MEMS composedof a light reflector and a support element that is controlled withtorsion and rotation through electromagnetic induction to support thereflector rotatably is further preferably used as the scanningmechanism.

[0026] An MEMS producing technology has been recently giving attentionon its practicality, especially in the monitoring apparatus of thepresent invention, by using the MEMS of an electromagnetic inductioncontrol without the need for the above described turnable part as thescanning mechanism, such problems as failures due to occurrence of heatof friction and metallic fatigue happened when using the polygon millerand the like, and a life of the monitoring apparatus is expected to beincreased.

[0027] The monitoring apparatus of the present invention, as describedabove, acquires position information on the incoming object, foracquiring more detailed position information, preferably comprises theirradiating device as the irradiating means and the imaging device asthe imaging means, wherein the position measuring means calculatesthree-dimensional position coordinates of the incoming object into thespace to be monitored based on the principle of triangulation techniquesby using position relations between the irradiating device and theimaging device, known shapes of the pattern light and displacementsacquired from image data captured by the imaging device when theincoming object entered into the space to be monitored and the standardimage data which was captured in advance.

[0028] The calculating manner described above shows merely the preferredexemplary, and it is our intention that the foregoing description doesnot limit the calculating manner for the position information of thepresent invention only to the calculating manner based on the principleof triangulation techniques.

[0029] In another preferred manner of the monitoring apparatus of thepresent invention for acquiring more detailed position information, theimaging means and the irradiating means are positioned and placed in amanner that an angle which a virtual line connecting the imaging meansand the irradiating means forms with a horizontal surface becomesapproximately 45°, so that the position information on an incomingobject is measured by a parallax.

[0030] According to the manner described above, it becomes possible togenerate positively ‘parallax’ which is approximately equal in thedirections of the height and the width, so that the measurement by theposition measuring means is performed more accurately.

[0031] The preferred embodiment of the monitoring apparatus of thepresent invention further comprises a means for specifying area selectedfreely as a specified area through a user's operation from the space tobe monitored divided into a plurality of areas virtually in advance andalso for selecting type of output selected freely through the user'soperation from a plurality of types of outputs prepared in advance atevery specified area, wherein the position measuring means acquiresspecified information on the area where the incoming object exists, andthe output-decision means outputs device control signal based on thetype of output which is set for the area specified through the positionmeasuring means.

[0032] As for the expression of “a plurality of types of outputsprepared in advance”, for example, an output of device stopping signal,an output of lighting signal such as warning signal, an output ofwarning operation signal and the like can be provided. Of course, othervariety of types of outputs can be further provided and prepared inadvance.

[0033] The expression of “specified information” means “information forspecifying the appropriate area”, for example, such information onserial numbers added to every area divided into a plurality of areas.

[0034] According to the foregoing manner, device control signalsprovided to every appropriate area based on the presence or absence ofthe incoming object into each area can be outputted automatically.

[0035] More preferably, the setting for the types of outputs isperformed by voxel that is partitioned based on the shape or the patternof the pattern light.

[0036] The expression of “voxel partitioned based on the shape or thepattern of the pattern light” means, as an example for the pattern lightshaped like a grid, the voxel partitioned by a grid line and apredetermined virtual cross section.

[0037] According to the manner described above, it becomes easy for auser to recognize (specify) areas divided virtually, so that it becomeseasy to set the types of the outputs.

[0038] In the manner described above, it is advisable for the specifiedarea to image an object in a state of placing the object at desiredposition in the space to be monitored and to be performed an automaticspecifying by extracting an area where the object exists.

[0039] The automatic specifying is appropriate to so-called ‘teaching’,but according to the foregoing manner, it is possible for an areaspecifying to be performed with avoiding intuitive ambiguity, withouterror and with reliability.

[0040] The preferred another embodiment of the three-dimensionalmonitoring apparatus of the present invention further comprises a meansfor specifying an area selected freely as a specified area through auser's operation among the space to be monitored divided into aplurality of areas virtually in advance and for selecting by specifiedarea types of outputs selected freely through the user's operation amongthe plurality of the types of outputs prepared in advance, wherein theposition measuring means measures a moving state of the incoming objectbased on the time series changes of a plurality of monitoring image datacaptured sequentially, the decision-output means specifies an predictivearrival area of the incoming object based on the measurement result andoutputs a device control signal based on the predictive arrival area andthe types of outputs determined in the predicted area.

[0041] As for the expression of “measures a moving state of the incomingobject”, a direction of moving, a moving speed and the like areconsidered as ‘a moving state’.

[0042] In the expression of “specifies an predicted arrival area”,meanings of the word ‘time’ which is a reference for the specifying isnot added any limitation, that is to say, it is considered for theexpression of the word ‘time’ to be in a manner to specify (predict) anobject-arrival area after a defined time (for example, ‘after onesecond’), or to be in a manner to include a plurality of areas at whichthe object is predicted to arrive within a predetermined time andrespectively calculate predicted arrival times at each area for addingthe times to a reference of decision for outputs.

[0043] According to the manner described above, it becomes possible toperform device control based on the predictions of arrivals of theincoming object to the predetermined areas, it becomes possible tooutput the control signal by a quick reaction to the incoming objectmoving at a high speed.

[0044] Next, a self-diagnosis function may be added to thethree-dimensional monitoring apparatus of the present invention. Forexample, the function can be added to the three-dimensional monitoringapparatus by providing a self-diagnosis means for recognizingappropriately whether projection pattern based on pattern lightirradiated via an irradiating means is matched with predicted projectionpattern and for ensuring normal operations of each configuration meansat the moment when the matching is recognized.

[0045] As another self-diagnosis function for the monitoring apparatusof the invention, a function for detecting automatically the presence orabsence of unusual conditions in own operations of the apparatus itselfbased on whether matching of the specified parts is recognized or not bychecking standard image data against monitoring image data captured whenthe self-diagnosis is performed can be recommended.

[0046] As other example of the self-diagnosis function, a function fordetecting automatically the presence or absence of unusual conditions inits own operations based on whether an predicted checked pattern isappeared normally in the image data acquired by irradiating directly apart of light irradiated from the irradiating means can be alsoconsidered.

[0047] In any manners described above, the self-diagnosis function isestablished, so that error operations for detection due to failures andthe like of the monitoring apparatus itself are prevented.

[0048] Now considering in an embodiment of the present invention, itseems preferable for the ‘self-diagnosis’ to be performed beforeprocessing of monitoring operations of the three-dimensional monitoringapparatus. More specifically, for example, a means for repeatingalternatively (Self-diagnosis→monitoring→diagnosis→monitoring, etc.) onetime processing of a self-diagnosis and one time processing of amonitoring operation (for example, processing of one time of imagecapturing is defined as one time processing) and a means for performingone time processing of self-diagnosis when starting up the monitoringapparatus then for monitoring repeatedly(self-diagnosis→monitoring→monitoring, etc.) can be considered in theembodiment. During the ‘self-diagnosis’, if an unusual condition isfound in any one of configuration elements such as the irradiatingmeans, the scanning means and the imaging means of the three-dimensionalmonitoring apparatus, it is preferable for the monitoring apparatus todisable irradiating a predetermined pattern light and stop starting ofthe monitoring operations, or it is possible to output a predeterminedoutput to inform NG as a result of the self-diagnosis to outside.Simultaneously, it is possible to bring a device to be monitored intosuch safety processing as stopping them.

[0049] Moreover, a plurality of the monitoring apparatuses describedabove can also be used simultaneously to monitor the space to bemonitored from a plurality of directions.

BRIEF DESCRIPTION OF THE DRAWINGS

[0050]FIG. 1 is an illustration of the whole configuration of athree-dimensional monitoring apparatus of the present invention.

[0051]FIG. 2 is a schematic illustration of a configuration and contentsof operations of a light source unit.

[0052]FIG. 3 is an illustration of a configuration of an MEMS adapted asa scanning mechanism.

[0053]FIG. 4 is an illustration of a schematic configuration of a cameraunit.

[0054]FIG. 5 is a block diagram of a configuration of a controller unit.

[0055]FIGS. 6A and 6B are an illustration of relations between a swingangle of an MEMS and a monitor range and a detecting resolution.

[0056]FIG. 7 is a schematic illustration of contents of a split of voxelin a space to be monitored.

[0057]FIGS. 8A and 8B are an illustration of a position relation betweenthe light source unit and the camera unit.

[0058]FIG. 9 is a flowchart of contents of a setting mode for space tobe monitored.

[0059]FIG. 10 is an illustration of a setting manner of a light sourceelement for monitoring in the light source unit.

[0060]FIG. 11 is a flowchart of contents of a calibration mode.

[0061]FIGS. 12A, 12B, 12C and 12D are a timing chart of a relationbetween swing angle control of the MEMS and control of oscillation andimaging timing of a laser light source element.

[0062]FIG. 13 is an illustration of a combined pattern projection of onecycle of a character 8.

[0063]FIGS. 14A and 14B are an illustration of already known angles θxand θy which a light axis forms with each grid line of gird patterlight.

[0064]FIG. 15 is a flowchart of contents of data storage processing.

[0065]FIG. 16 is an illustration of an example of a data structure of agrid line.

[0066]FIGS. 17A and 17B are an illustration of an example of types ofoutputs setting to the space to be monitored.

[0067]FIG. 18 is an illustration of an example of an interface (a screenof a personal computer) to set the space to be monitored.

[0068]FIGS. 19A and 19B are an illustration of contents of a voxelautomatic specifying (teaching) associated with the types of outputssetting.

[0069]FIG. 20 is a schematic illustration of an example usingsimultaneously two pair of the light source and the camera unit.

[0070]FIG. 21 is a flowchart of contents of self-diagnosis processing.

[0071]FIGS. 22A and 22B are the first illustration of contents of imageprocessing by an image processing measuring part.

[0072]FIGS. 23A and 23B are the second illustration of contents of imageprocessing by an image processing measuring part.

[0073]FIGS. 24A and 24B are the third illustration of contents of imageprocessing by an image processing measuring part.

[0074]FIG. 25 is a schematic illustration of distance-measurementprinciple based on the triangulation techniques adapted to the presentinvention.

[0075]FIG. 26 is the first flowchart of contents of processingassociated with distance-measurement.

[0076]FIG. 27 is the second flowchart of contents of processingassociated with distance-measurement.

[0077]FIG. 28 is the third flowchart of contents of processingassociated with distance-measurement.

[0078]FIGS. 29A and 29B are an illustration of contents of processing ofan arrival range prediction of an object in the space to be monitored.

[0079]FIG. 30 is a flowchart of contents of processing of the arrivalrange prediction of the object.

[0080]FIG. 31 is a block diagram of a configuration of an apparatus forthe self-diagnosis processing in another embodiment.

[0081]FIGS. 32A, 32B, 32C and 32D are a timing chart of contents ofapparatus control for the self-diagnosis processing in anotherembodiment.

[0082]FIGS. 33A and 33B is an illustration of an example of capturedimage associated with the self-diagnosis processing in anotherembodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0083] The following description provides a detailed explanation on apreferred embodiment of the three-dimensional monitoring apparatus ofthe present invention in reference to attached drawings.

[0084] Referring first to the consideration of FIG. 1, FIG. 1illustrates the whole of a configuration and an outline of operations ofthe three-dimensional monitoring apparatus in the embodiment. Asillustrated, a three-dimensional monitoring apparatus 100 comprises alight source unit 1 as an irradiating means, a camera unit 2 as animaging means and a controller 3 to control the overall units; a lightsource unit 1 and a camera unit 2.

[0085] The light source unit 1 irradiates a predetermined laser patternlight (hereafter referred to as a pattern light) to a three-dimensionalspace S to be monitored (hereafter referred to as a monitoring space).In the embodiment, a screen 5 is placed at the rear of the monitoringspace S, a projection pattern (in the embodiment, an infrared projectionwhich is not recognized by human eyes) based on the pattern lightirradiated from the unit light source 1 is displayed on the screen 5.

[0086] When incoming an object such as a human or a matter (hereafterreferred to as an incoming object M) into the space S, the infraredpattern projection displayed on the screen 5 is changed. With imagingthe changing pattern image by the camera unit 2 and performing imageprocessing, the position and the moving direction of the incoming objectM (in FIG. 1, a human is illustrated) are detected.

[0087] The controller 3 controls the light source unit 1 and camera unit2, and also performs outputs according to the detected position and themoving direction of the incoming object M in the monitoring space S,more specifically, in the embodiment, outputs light on-off controlsignals (in the embodiment, ‘stop operation (red)’, ‘warning (yellow)’and ‘no reaction (blue)’ to a signal lamp 6 and also outputs controlsignals ‘operation start (ON)’ and ‘operation stop (OFF)’ to a facilitydevice 4 to be controlled. It is decided which signal is outputtedaccording to the detected position and the moving direction of theincoming object M based on the setting of types of outputs set freely bythe user through the controller 3.

[0088] Referring to FIG. 2 illustrating details on the light source unit1, the light source unit 1 comprises a light source element 11 tooscillate an infrared laser beam, an MLA (micro lens array) 12 as apattern light generator to convert the infrared laser beam projectedfrom the source element 11 into a predetermined pattern light and a MEMS13 as a scanning mechanism having a miller plane (see FIG. 3) under anoperation control (a vibration control) by electro-magnetic inductionand conducting the pattern light generated by the MLA 12 to any positionin the monitoring space S. In FIG. 1, reference symbol Ls indicates alight axis of the pattern light (a virtual expanded line of a laser beamlight axis projected from the light source element 11).

[0089] Intensity of the infrared laser beam irradiated from the lightsource element 11 is controlled not to exert adverse effects to a humanbody and the infrared laser beam is converted into pattern lights(patterns are illustrated as referred symbols A21-A25 in FIG. 2) shapedlike a grid (shaped like a net). By reflecting the pattern light in thedirection of a predetermined direction through the miller plane of theMEMS 13, the pattern light is scanned in the monitoring space S. Thecontroller 3 performs oscillation controls and ON/OFF controls of thelight source element 11 and operation controls of the MEMS 13.

[0090] As described later, the light source unit 1 furthermore comprisesa light source element 14 for monitoring (a visible laser beam lightsource) and a half miller 15 to set the monitoring space S (see FIG.10).

[0091] Referring to FIG. 3, details of the MEMS 13 is described asfollows. In FIG. 3, torsion bars (torsion axes) indicated as referencenumbers 134 and 135 are illustrated, around the torsion bars 134 and 135a silicon board (not illustrated) is further connected (see MEMS 134 inFIG. 2), and reference numbers 136 and 137 in FIG. 3 indicate permanentmagnets for electromagnetic induction.

[0092] The MEMS 13 used in the embodiment has a miller plane 131arranged on a silicon substrate 130 and a coil pattern illustrated witha sequential line in FIG. 3 is formed on a surface of the siliconsubstrate 130 on which the miller plane 131 is mounted. Vibrationcontrol is enabled in the directions of x-axis and y-axis illustrated inFIG. 3 within a predetermined angle range with rotating axes of torsionbars (both hands of the silicon substrate 130) referring to numbers132,133,134 and 135 in FIG. 3.

[0093] More specifically, while magnetizing a magnetic field B in thespecific directions BA and BB, when feeding currents i (iA and iB)through a coil, a rotation torque is generated by Lorentz forces F (FAand FB), the silicon substrate 130 (the miller plane 131) can bevibrated up to the position where the forces F match with restoringforces of the torsion bars (torsion axes) 132,133,134 and 135. Accordingto the MEMS 13, by the torsion bars 132 and 133 in the direction ofx-axis and torsion bars 134 and 135 in the direction of y-axis, theirradiating direction of the pattern light can be controlled freely.

[0094] Then, in the embodiment, the light source unit 1 is configured asdescribed above, so that following effects a) and b) are produced.

[0095] a) It becomes possible to generate a variety of pattern lightsuch as shaped like circle and like concentric polygon not limited tothe pattern light shaped like a grid by changing MLA.

[0096] b) In the MLA 12 and the MEMS 13, there is no such movable partas a polygon miller generating friction, so that expansion of life ofthe monitoring apparatus can be expected.

[0097] As a pattern light generator, a CGH (computer graphics hologram)can be adapted instead of the MLA 12, and in this case, the same effectsa) and b) described above can be produced.

[0098] Referring to FIG. 4, a configuration of the camera unit 2 will beexplained. As shown in FIG. 4, The camera unit 2 is composed of a camera21 including an imaging device 21 a and a shutter mechanism 21 b and ofa band path filter (BPF) 22 corresponding to wavelengths of an infraredbeam irradiated from the light source element 11, projection patternsprojected on the screen 5 and projected on the incoming object M areimaged through a shutter timing control by the controller unit 3. In theembodiment, a CCD camera is used as the camera 21, so that the mechanism21 b is not existed actually, but for the explanation of the camera unit2, the mechanism 21 b is illustrated in FIG. 4.

[0099] In the embodiment, the band path filter 22 prevents from incomingof external fluctuated light such as fluorescent lamp light andsunlight.

[0100] A block diagram in FIG. 5 illustrates a configuration of thecontroller unit 3. As shown in FIG. 5, the controller unit 3 comprises asetting control part 30, a light source control part 31, a camera unitcontrol part 32, an image processing measuring part 33, a self-diagnosispart 34 and a decision part 35.

[0101] The controller unit 3 illustrated in FIG. 5 comprises two inputsindicated as ‘a signal input for a user setting IN1’ and ‘a signal inputfor a camera image IN2’ and four outputs indicted as ‘a signal outputfor a device control OUT1’, ‘a signal output for a control signal for alight ON/OFF OUT2’, ‘a signal output for a control signal for a lightsource unit OUT3’ and ‘a signal output for control signal for a cameraunit OUT4’.

[0102] The setting control part 30 switches operation modes of thethree-dimensional monitoring apparatus 1000 based on the user settinginput IN1 and also a variety of setting values (data) acquired throughthe user setting input IN1 into a predetermined memory.

[0103] In the embodiment, four modes are prepared as the operation modesas follows.

[0104] 1) A Monitoring Mode

[0105] (Contents) In a monitoring mode, a position or a moving directionof the incoming object M in the monitoring space S are detected togenerate an output corresponding to a detecting result.

[0106] 2) A Setting Mode for Space to be Monitored

[0107] (Contents) In a setting mode for space to be monitored, a rangeof the monitoring space S is set.

[0108] (Detailed description) FIG. 6 shows relations between a vibrationangle of the MEMS 13, and a monitor range and a detecting resolution.The light source unit 1 scans the pattern light shaped like a grid inthe monitoring space S by operation control of the MEMS 13. As shown inFIG. 6A, if the vibration angle (hereafter called a swing angle) θ ofthe MEMS 13 (the miller plane 131) is set at wide (shown as θwx61) toextend a scanning range (shown as a projection range of a combinedprojection pattern indicated by a reference number A61 in FIG. 6), arelative wide range is permitted to be monitored.

[0109] On the other hand, as shown in FIG. 6B, if the swing angle θ isset at narrow (shown as θwx62), to narrow the irradiating range (shownas a projection range of the combined projection pattern indicated by areference symbol A62 in FIG. 6), a density of grid lines (the number ofgrid lines per a unit area) becomes in high and detection with arelative high resolution is permitted.

[0110] In the setting mode for space to be monitored, the swing angleθwx (or θwy) is accepted through a predetermined user's operation.

[0111] In the embodiment shown in FIG. 6, the MEMS 13 scans (swings) thepattern light only in the direction of the x-axis (around of they-axis), but as mentioned above, the MEMS 13 adopted in the presentembodiment can scan the pattern light in both direction of the x-axisand the y-axis.

[0112] 3) A Calibration Mode

[0113] (Contents) In a calibration mode, distance data L (refer FIG. 8A)is acquired, and the pattern light is irradiated during a state that theincoming object M does not exist in the monitoring space S then theprojection pattern projected on the screen 5 is imaged by the cameraunit 2 to capture a standard image (hereafter called a reference image).

[0114] (Detailed description) In the embodiment, as will be describedlater, a distance up to the incoming object M is calculated by theprinciple of triangulation techniques based on the comparison between animage (hereafter called a monitoring image) which is imaged by thecamera unit 2 in the monitoring mode and the reference image. Thedistance data L calculated in the calibration mode is used for aself-diagnosis described later.

[0115] 4) Setting Mode for Types of Outputs

[0116] (Contents) In a setting mode for types of outputs, outputscorresponding to the position and the moving direction of the incomingobject M in the monitoring space S are outputted.

[0117] (Detailed description) The monitoring space S whose rage is setin the setting mode for space to be monitored is partitioned into aplurality of virtual voxels (Volume Pixel)(4×4×3 pieces in theembodiment). In the embodiment, it is possible to preset any types ofoutputs by voxel (or by a plurality of voxels) according to the presenceor absence of the incoming object into each voxel.

[0118] In the embodiment, three kids of outputs such as ‘stop’,‘warning’ and ‘no reaction’ are prepared as the types of outputs. Asetting manner will be described later, types of outputs setting datawhich is set in the setting mode for types of outputs is used by adecision part 35, and as for information, a default value for thesetting is defined as ‘warning’ as shown in FIG. 6B.

[0119] Grid patterns shown as reference symbols A71-A74 in FIG. 7indicate conceptual virtually combined pattern projection recognized ineach divided surface splitting the monitoring space S into three inforward and backward directions (in the embodiment generating combinedpattern projection by scanning the pattern light, the combined patternprojection is a combined 4×4 pattern projection). A voxel Vp shown inFIG. 7 indicates a voxel corresponding to a space between plane areasP72 and P73 partitioned by combined pattern projections A72 and A73among a central space S2 splitting the monitoring space S into three inforward and backward directions.

[0120] Referring again to FIG. 5, the light source unit control part 31controls operations (swing angles) of the MEMS 13 based on the swingangle defaults (θwx and θwy) stored into the setting control part 30 andalso controls ON/OFF and oscillations of the light source element 11(and a light source element 14 for monitoring which will be describedlater).

[0121] The camera unit control part 32 performs a shutter timing controland an image capturing control of the camera unit 2 to capture themonitoring image and the reference image. Both of the images are storedas binary data into a frame memory F1 or F2 of the image processingmeasuring part 33 which will be described later. The shutter timingcontrol is performed in reference to a state of an oscillation (anoscillation control signal) of the light source unit control part 31 anda state of an operation (a swing angle control signal) of the MEMS 13.

[0122] The image processing measuring part 33 measures a existingposition (space coordinates z (x, y)) of the incoming object M by usingthe principle of triangulation techniques based on the monitoring imageand the reference image captured by the control part 32 to output ameasurement result to the decision part 35. The image processingmeasuring part 33 is provided with the frame memory FM1 (for themonitoring image) and FM2 (for the reference image) storing image datacaptured by the camera unit control part 32.

[0123] The self-diagnosis part 34 diagnoses whether thethree-dimensional monitoring apparatus 100 works normally or not, morespecifically, diagnoses whether the laser unit control part 31 and thecamera unit control part 32 work normally based on the image datacaptured through the camera unit control part 32 every time the imagedata is captured. The diagnosis result is inputted to the decision part35.

[0124] The decision part 35 decides the signal output for light on-offcontrol OUT2 among the three types of signal outputs OUT1-OUT3 anddecides simultaneously the signal output for device control OUT1 to theequipment device 4 based on the result from the measuring part 33, thesetting of the space S, the setting of the types of outputs by voxel Vpand the diagnosis result from the self-diagnosis part 34.

[0125] Furthermore, it is also possible for the decision part 35 todecide the signal output for device control OUT1 to be ‘stop ofoperation in advance when an incoming of the incoming object M isobvious by predicting the presence or absence of the incoming object Minto a predetermined voxel which was specified in advance according tothe moving speed and moving direction of the incoming object M (apredictive stop function), and the predictive stop function will bedescribed later.

[0126] The three-dimensional monitoring apparatus 100 in the embodimentis described step-by-step as follows.

[0127] A position relation between the light source unit 100 and thecamera unit 2 in the three-dimensional monitoring apparatus 100 in theembodiment is illustrated diagrammatically in FIG. 8. As shown in FIG.8A, the light source unit 1 and the camera unit 2 are placed nearby eachother in a manner that each front plane (in precise, an MEMS plane forthe light source unit 1 and a camera lens plane for the camera unit 2)is positioned equidistant from the screen 5 (indicated as a distance Lin FIG. 8). Furthermore, the position relation between light source unit1 and camera unit 2 simultaneously fills following conditions a)-c).

[0128] a) A light axis of the light source unit 1 (indicated as a dashedline J1 in FIG. 8) is parallel to a light axis of the camera unit 2(indicated as a dashed line J2 in FIG. 8). The light axis J1 of thesource unit 1 is a normal line of the MEMS plane and the light axis J2of the camera unit 2 is a normal line of a lens plane.

[0129] b) Grid lines in a lateral direction of the pattern lightirradiated from the light source unit 1 are parallel to a scanningdirection of the x-axis (a scanning direction of an imaging element CCD(charge coupled device) of the camera unit 2) (details will be describedlater) of image data surrounded by the measuring part 33.

[0130] c) The camera unit 2 is placed at the position in a slantingdownward direction with an angle 45° of the light source unit 1(indicated as an angle of θ80 in FIG. 8B).

[0131] The conditions described above are the setting conditionspredicted on a applying of a calculation equation for object positionmeasurement that will be described later, and the setting conditionsdescribed above are not to be considered to limit the position relationbetween the light source unit 1 and the camera unit 2.

[0132] As a postscript on the foregoing condition c), since the cameraunit 2 is designed to be placed at the position in the slanting downwarddirection with the angle 450 of the light source unit 1, a parallax canbe evenly generated to both of the x-axis and the y-axis. For example,even if the camera unit 2 is placed in a upper right, an upper left or alower right direction of the light source unit 1, same effects can beproduced.

[0133] Operations of the three-dimensional monitoring apparatus in thesetting mode for space to be monitored are shown in a flowchart in FIG.9.

[0134] The setting mode for space to be monitored is started up underthe condition that the setting mode for space to be monitored isspecified through the signal input for a user setting IN1 (YES, in step901, step 902). If other mode is set by the signal input for a usersetting IN1 and the like (No, in step 902), the mode is changed to modescorresponding to each specified mode (step 903).

[0135] When the setting mode for space to be monitored is started up,the light source element 14 for a monitor on FIG. 10 starts up a laserbeam oscillation with a control by the light source unit control part31, and simultaneously, the MEMS 13 is vibrated in the both directionsof the x-axis and the y-axis based on the setting values of swing anglesθwx and θwy stored into a predetermined memory in the setting controlpart 30(step 905). Only for setting in the first time, defaults aretaken as setting values of the swing angles θwx and θwy.

[0136] Having been omitted in FIG. 2, but as shown in FIG. 10, the lightsource element 14 for a monitor is a light source element for a visiblelaser beam oscillation arranged in the light source unit 1, a laser beamirradiated from the light source element 14 is irradiated to the MLA 12with the same light axis Ls as a laser beam irradiated from the halfmiller 15 placed between the light source element 11 and the MLA 12.

[0137] Accordingly, visible pattern light having the same pattern thatof the pattern light from the light source element 11 is scanned todisplay a visible combined pattern projection is displayed on the screen5, and the user can recognize the setting range of the monitoring spaceS.

[0138] Next, when the user operates a lever switch (an operating part)which is not shown in FIG. 10 (YES, in step 906), the swing angle ischanged according to the operation (step 907), the combined patternprojection (a scanning range of the pattern light) displayed on thescreen 5. After operating the lever switch, when the user pushes adecision button which is not illustrated in FIG. 10, swing angles atthat time of pushing are overwritten and stored as new swing anglessetting values θwx and θwy into the predetermined memory in the settingcontrol part 30 (step 909), then processing is completed temporarily.

[0139] In the present embodiment, therefore, user can set the monitoringspace S in accordance with the visible combined pattern projection inthe setting mode for space to be monitored.

[0140] Operations of the three-dimensional monitoring apparatus in thecalibration mode are shown in a flowchart in FIG. 11.

[0141] The calibration mode is started up through the signal input for auser setting IN1 under a condition that the calibration mode isspecified (step 1101), processing becomes into a state of waiting for aninput of the distance data L already described (NO, in step 1102, step1103). When the user inputs the distance data L through a key operatingpart (for example a ten key) of the controller 3 (YES, in step 1102),the distance data L is stored into the predetermined memory in thesetting control part 30 (step 1104).

[0142] Next, the swing angle setting values θwx and θwy of the MEMS 13are stored into the setting control part 30 (step 1105).

[0143] The light source unit control part 30 performs laser oscillationcontrol of the light source element 11 and performs swing angle controlof the MEMS 13 based on the read swing angle setting values θwx and θwyto project the combined pattern projection on the screen 5 (step 1106).

[0144] The camera unit control part 32 controls the camera unit 2 toimage the combined pattern projection projected on the screen 5 (step1107) and encodes the captured image data in binary form to store intothe frame memory FM2 in the image processing measuring part 33(step1108). As mentioned above, the reference image is captured in thecalibration mode only in the state of the absence of the incoming objectM in the monitoring space S.

[0145] In the flowchart in FIG. 11, when controlling the swing angles,the control part 30 outputs to the source unit 1 the control signalshown in the timing chart in FIG. 12.

[0146] In FIG. 12A shows a control signal regarding to the swing anglein the direction of the x-axis θwx, FIG. 12B shows a control signalregarding to the swing angle in the direction of the y-axis θwy, FIG.12C shows an oscillation control signal (ON/OFF control signal) of thelight source element 11 and FIG. 12D shows instant pattern projectionsA121-A124 (shown in FIG. 13) projected at respective timings.

[0147] In the calibration mode, the light source unit control part 31outputs to the light source unit 1 swing angle control signals of theMEMS 13 shown in FIG. 12A and FIG. 12B and a light ON/OFF signal to thelight source element 11 shown at FIG. 12C. In FIG. 13, patternprojection indicated as a reference symbol A120 shows a combined patternprojection projected on the screen 5.

[0148] The camera unit control part 32 monitors the control signal fromthe light source control part 31 and performs open-close control to theshutter mechanism 21 b of the camera unit 2 to record one cycle of acharacter 8 (four instant images in the embodiment) as a single image.The image processing measuring part 33 stores the single image of onecycle of a character 8 into the frame memory FM2 for reference image.

[0149] Next, the image processing measuring part 33 calculates an angleθ which each grid line of a vertical direction (the direction of they-axis) and a horizontal direction (the direction of the x-axis) of thecombined grid pattern projection forms with the z-axis (a light axis ofthe laser beam).The angle θ is used to calculate a distance up to theincoming object M.

[0150] Hereby, a distance between the light source element 11 of thelight source unit 1 and the MLA 12 is already known, angles (shown inFIG. 14A as θx1-θx4) which the z-axis forms with each vertical grid lineof a basic grid pattern projection (shown as 4×4 grid pattern light inthe embodiment) and angles (θy1-θy4) which the z-axis forms with eachhorizontal grid line are also known from a design viewpoint. A swingangle control signal of the MEMS 13 and swing angles (setting values) ofthe MEMS 13 shown in FIG. 12A and FIG. 12B are also already known.(Consequently, the combined image projection shown in FIG. 13 isprojected by swinging the swing angles of the MEMS 13 up to −θw/2 to+θw/2 in the direction of the x-axis and up to −θw/2 to +θw/2 in thedirection of the y-axis.

[0151] If the MLA 12 generating the grid pattern light shown in FIG. 14,an angle θx which a vertical grid line at the left end of the combinedpattern projection forms with the z-axis shown in FIG. 13 is expressedin an equation θx=−θw/2−θx1. Similarly, an angle that each vertical andhorizontal grid line forms with the z-axis can be calculated from swingangles of the MEMS 13 and information already known from a designviewpoint.

[0152] Hereby, for the sake of clarity, pattern light shaped like a 4×4grid is described as an example, but even after increasing of grid innumber, these angles can be calculated similarly.

[0153] For the next, coordinates (xf, yf) on image elements (CCDs) ofeach vertical and horizontal grid line are retrieved from the datastored into the frame memory FM2 to store a retrieval result in, forexample, a data structure in a C language is as follows.

[0154] [Equation 1] typedef struct grid_v{/*a data structure forvertical grid line storage*/ int angle/* an angle from the z-axis*/ inty_min; /*a minimum value of Y*/ int y_max; /*a maximum value of Y*/ intxf [MAX_YSIZE]; /* x coordinates of grid lines on a frame memory*//*MAX_YSIZE:a maximum y coordinates on a frame  memory*/ int yf[MAX_YSIZE]; /* y coordinates of grid lines on a  frame memry*/}GRID_V;typedef struct grid_h{/*a data structure for horizontal grid linestorage*/ int angle/* an angle from the z-axis*/ int x_min; /*a minimumvalue of X*/ int x_max; /*a maximum value of X*/ int xf[MAX_XSIZE]; /* xcoordinates of grid lines on a frame memory*/ /*MAX XSIZE:a maximum Xcoordinates of a frame memory* / int ty[MAX_XSIZE]; /* y coordinates ofgrid lines on a frame memory*/}GRID_V; GRID_V Lv [GRIDMAX]; GRID_H Lh[GRIDMAX];

[0155] Detailed procedures of data storage processing described aboveare shown in a flowchart in FIG. 15. As an previous description, FIG. 16shows an example of a data structure stored into the data storageprocessing (the example is a data structure in the case of imaging of a6×7 combined image pattern projection).

[0156] In a data storage processing, at first, vertical components aredetected in filtering to detect vertical grid lines (step 1501).

[0157] Next, edge lines are expanded and reduced to repair vertical gridlines which are cut during image processing (steps 1502 and 1503) toenhance vertical components.

[0158] Furthermore, coordinates on such image as shown in FIG. 16 (inthe example, (0, 0) to (MAX_XSIZE−1, MAX_YSIZE−1) are scanned to detectvertical grid lines. At every detected gird line, the coordinates, themaximum value and the minimum value of the Y coordinates and the angleθx from the z-axis calculated by the previous calculation of the gridangles are stored into an appropriate data structure Lv [n] (step 1504).In the present example, the number ‘n’ shown in the data structure Lv[n] is incremented by +1 as it heads for the right from the left end ofthe grid lines which is set at ‘0’.

[0159] Similarly, horizontal components are detected in filtering todetect horizontal grid lines (step 1505).

[0160] For the next, edge lines are expanded and reduced to repairhorizontal grid lines which are cut during the image processing (steps1506 and 1507) to enhance vertical components.

[0161] Furthermore, coordinates on such image as shown in FIG. 16 (inthe example, (0, 0) to (MAX_XSIZE−1, MAX_YSIZE−1) are scanned to detecthorizontal grid lines. At every detected gird line, the coordinates, themaximum value and the minimum value of the X coordinates and the angleθx from the z-axis calculated by the previous calculation of the gridangles are stored into an appropriate data structure Lh [n] (step 1508).In the present example, the number ‘h’ shown in the data structure Lh[n] is incremented by +1 as it heads downward from the left end of thegrid lines which is set at ‘0’.

[0162] The following description will explain the details on the settingmode for types of outputs. In the setting mode for types of outputs, thetypes of outputs deciding signal outputs OUT1 and OUT2 to the equipment4 and the signal lamp 6 are selected when the incoming object M such asa human body or an obstacle enters into the monitoring space S.

[0163] As already shown in FIG. 7, setting for outputs can be performedat every voxel. In FIG. 7, though a 4×4×3 voxel is set, a split of avoxel can be also set freely without reference to the pattern light.

[0164] An example of the types of outputs setting for the monitoringspace S is shown in FIG. 17. In an example of the setting shown in FIG.17A, hatched areas (voxels S131 and S132) are set to ‘stop operation’.If the human or the obstacle enters to the areas, the signal lamp 6comes on in a red and the equipment 4 comes to a stop of an operation.Areas in white except for hatched areas are set to ‘warning’, if theobject M enters into the areas, the lamp 6 comes on in a yellow tooperate the equipment 4 without interruption.

[0165] There are occasions when an operator wants to enter into themonitoring space S to adjust the equipment, in this situation, it isdangerous for the operator to be pushed from the rear by an incoming ofanother operator (or an obstacle) during the adjustment. In such case,it is possible for the operator to be in safety and to increaseproductivity with the setting of the type of output to ‘no reaction’ inthe area needed for operations. In an example shown in FIG. 17B, whenthe human or the obstacle enters into areas indicated by voxels S132 andS133, the signal lamp 6 comes on in a red and the equipment 4 subject tostop its operation, but an area indicated by a voxel S134 is set to ‘noreaction’, so that if the human or the object enters into the voxelS134, the output is not changed.

[0166] An example of an interface to set the monitoring space S is shownin FIG. 18. In the example, the interface is provided with software on apersonal computer. On a display 140 of the computer, a projection for abird's eye view 18 a allowing a bird's eye view by splitting themonitoring space S into voxels, a projection for X-Z selection 18 b toselect a voxel by watching the monitoring space S from a direct side, aprojection for Y-X selection 18 c to specify a voxel by watching themonitoring space S from a right front and a projection for setting typeof output 18 d to set types of outputs by voxel are displayedsimultaneously.

[0167] User setting procedures by using the interface are described asfollows.

[0168] 1. To specify an X-Z area to which a voxel desired to be setbelongs on the projection for X-Z selection 18 b by a cursor (referencesymbol B in FIG. 18.

[0169] 2. To specify an X-Y area to which a voxel desired to be setbelongs on the projection for X-Y selection 18 c so as to complete aspecifying of a voxel (reference symbol C in FIG. 18).

[0170] 3. To select a type of output from the projection for settingtypes of outputs 18 d for the specified voxel.

[0171] When completing the selection of the types of outputs, acorresponded voxel is indicated with a color at the projection forbird's eye view 18 a (a reference symbol A in FIG. 18).

[0172] 4. After that, procedures 1 to 3 are repeated to performnecessary setting for type of output to any voxel.

[0173] Another setting method for type of output (a voxel teaching) isshown in FIG. 19. In this example, the object M desired to be detectedsuch as a human or an obstacle was placed at any position in themonitoring space S in advance. In this situation, if a predeterminedsetting instruction is inputted, a voxel to which the object M belongsis selected, then, a desired type of output for the voxel is set by animage for setting a type of output.

[0174] In FIG. 20, another embodiment of the present invention isillustrated. In the foregoing embodiment, only one set of light sourceunit is used, so that it is impossible for an operator while operatingin a predetermined area in the monitoring space S to monitorsimultaneously the rear in which the pattern light irradiation isblocked. In the embodiment illustrated in FIG. 20, another one pair ofunits (a light source unit 1000 and a camera unit 2000) which is thesame as that of the pair of the light source unit 1 and the camera unit2 shown in the foregoing embodiment is used to irradiate an imagerespectively from different directions (in the embodiment, from a frontand a slanting upper directions of the screen 5), so that a blind spotis compensated and further operator safety is assured.

[0175] For the next, a content of operations in the monitoring mode,which is a main function of the three-dimensional monitoring apparatusof the present embodiment, is described as follows. Setting themonitoring mode to the setting control part 30 through a user'soperation performs the monitoring mode.

[0176] The three-dimensional monitoring apparatus 100 performs aself-diagnosis before starting the monitoring. At first, a method forself-diagnosis adapted to the present embodiment is described.

[0177] During the self-diagnosis, each unit operates as follows.

[0178] The control part 31 controls the light source unit 1 as is thecase of the calibration mode and scans the pattern light in themonitoring space S.

[0179] The camera unit control part monitors the control signal from thecontrol part 31 as is the case of the calibration mode and controls theshutter 21 b of the camera unit 2 to acquire image data of one cycle ofa character 8. When capturing an image from the camera unit 2, the framememory FM1 for reading images should be completely cleared in advance.

[0180] The self-diagnosis part 34 comprises image data for diagnosiscaptured from the camera unit 2 with data of the reference image storedinto the frame memory FM2, if the same image pattern is recognized inthe same area range, it is decided that the self-diagnosis part 34 givesits OK to proceed to the monitoring. If the image pattern is notrecognized or if an existence area of the captured image pattern iswider or narrower than that of the data of the reference image, it isdecided that the self-diagnosis part 34 gives its NG to perform themonitoring.

[0181] When intending to execute the self-diagnosis, the object M hasalready entered into the monitoring space S sometimes, in this case, ifan irregular spot in an image pattern is found within an outer frame(within an allowable area range) of a pattern of the reference image, itis considered that the irregular spot is caused by an entering of theincoming object M and the self-diagnosis part 34 gives its OK to proceedto the monitoring.

[0182] When the self-diagnosis part 34 has given its NG, the decisionpart 35 outputs the signal output OUT1 of ‘stop’ to stop the equipment 4immediately.

[0183] Referring to the flowchart in FIG. 21 for the detailed contentsof operations of the self-diagnosis, the embodiment indicated by theflowchart shows the case when the pattern projection is projected on thescreen 5 as already shown in FIG. 16.

[0184] When starting the self-diagnosis processing, at first, the framememory FM1 to capture images is cleared (step 2101). The projectionpattern of one cycle of a character 8 is imaged as is the case of thecalibration mode, and a self-diagnosis image is stored into the framememory FM1 (step 2102).

[0185] If there is a difference between the reference image pre-storedin the frame memory FM2 and the self-diagnosis image, it may beconsidered that the difference is caused by irregular occurrences ofgroups of light-irradiation, light receptive, scanning, or an occurrenceof an incoming of the incoming object M. In the embodiment, then thedifference between the reference image and the self-diagnosis image isextracted (step 2103).

[0186] If there is no difference between the reference image and theself-diagnosis image (FM1=FM1−FM2, YES, in step 2104), it is consideredthat the monitoring apparatus 100 works normally (go to a step 2107).

[0187] If there is the difference between the reference image and theself-diagnosis image (NO, in step 2104), it is decided whethercoordinates of a part causing the difference between the reference imageand the self-diagnosis image exist outside of an outer grid line of thereference image (outside of the range corresponding to Lh[0]to Lh[7], orLv[0] to Lv[6] shown in FIG. 16) or not (whether the coordinates existwithin an allowable range or not). If there are no coordinates withinthe allowable range (NO, in step 2105), there is a possibility that overscanning has occurred, so that the self-diagnosis part 34 gives its NG(step 2108) and the processing is ended temporarily.

[0188] If there is the difference within the acceptance range (YES, instep 2105) then the right end and the left end of the vertical grid line(Lv[0] and Lv[6]) are searched, and if the both ends are searchednormally (YES, in step 2106), it is considered that a control outputfrom the light source unit control part 31 can be matched with the imagepattern captured by the control output. Accordingly, in this case, it isconsidered that the difference between the both images is caused by theincoming of the incoming object M (YES, in step 2107), but if the bothends are not searched normally (NO, in step 2106), it is considered thatthere is some irregularity, the self-diagnosis part 34 gives its NG andthe signal output (OUT1) to stop the equipment 4 is outputted from thedecision part 35 (step 2108). It is possible for the normality of bothends Lv[0]and Lv[6] to be shown by conforming the absence of thedifference between the frame memories FM1 and FM2 on the coordinatesstored into the data structure.

[0189] The distance data L up to the screen 5 acquired in thecalibration mode is compared with the distance to the Lh[0](the top endof the horizontal grid line) acquired by the principle of triangulationtechniques. If the both distances are equal (including the case that thedifference between the both distances is within an allowable errorrange) (YES, in step 2107), it is considered that calculation is alsocompleted normally and the self-diagnosis gives the OK grade (YES instep 2107, step 2109). If there is difference between the both distances(NO, in step 2107), it is considered that something unusual has occurredand the self-diagnosis part 34 should give NG to output the signal tostop the equipment 4 (step 2108). A manner to calculate the distance upto the Lh[0] will be described later.

[0190] If the self-diagnosis part 34 gives OK, the monitoring isperformed continuously. During the monitoring, each unit works asfollows.

[0191] The light source unit control part 31 controls the light sourceunit 1 to scan the pattern light in the monitoring space S as in thecase of the calibration mode.

[0192] The camera unit control part 32 monitors the control signal fromthe light source unit control part 31 as in the case of the calibrationmode and controls the shutter 21 b of the camera unit 2 to acquire theimage data of one cycle of a character 8.

[0193] The image processing measuring part 33 calculates a differencebetween capture image shown in FIG. 22A (the camera unit 2 has a filtercorresponding to a wavelength of the light source element, so that anoutline of a human body is not projected actually as shown in FIG. 22)and the reference image stored into the frame memory FM2 to store adifference image shown in FIG. 22B into the frame memory FM1.

[0194] For the next, a distance map z (x, y) to the detected objects Mis calculated by using the original position of the grid line and adifference from the original position.

[0195] For the sake of clarity, the difference image stored into theframe memory FM1 is split to X-axis components and Y-axis components toshow in FIG. 23A and FIG. 23B.

[0196] As shown in FIG. 23, in the difference image, each pixel value ofparts (coordinates) which are not entered by the objects become 0values. If positions of the grid lines are moved by incoming of theincoming objects M, the pixel values of the parts of the original gridlines becomes minus values and coordinates of destination positionswhere the grid lines have moved become plus values.

[0197] Coordinates of each grid line are already stored into the dataconstruction (Lh[ ] and Lv[ ]) when generating the reference image (seeFIG. 15 and FIG. 16), so that moving destinations of each grid line aresearched along the horizontal grid lines and vertical grid lines basedon the data construction. The situation is shown in schematic form inFIG. 24

[0198] As shown in FIG. 24A, in the assumption that a line from a pointa to a point b indicates the original grid line, the grid line aftermoving is searched from a middle point c toward a slanting 45°direction. The search toward the slanting 45° direction is decided basedon the position relation between the light source unit 1 and the cameraunit 2 as already shown in FIG. 8.

[0199] If the moved grid line is found by searching (a point d in FIG.24), whole of coordinates of moved grid lines (e to d to f) is acquired.

[0200] Angles of the grid lines to the light axis Z of the light sourceelement 11 are already known (already stored into the ‘angle’ of thedata structure Lh[ ] and Lv[ ]), so that the distance map z (x, y) canbe calculated by the principle of triangulation techniques withcalculating angles between grid lines which is off from a light axisobserved on the CCD.

[0201] The principle of calculation by triangulation techniques applyingto the present embodiment is shown in FIG. 25. Now assuming that thenumber of pixels of the CCD is 640×480, the pitch of the pixel of theCCD is ccd_p, the light axis of the CCD passes (319, 239) and a focallength of the camera unit 2 is ‘1’.

[0202] Assuming that the coordinates of the grid line of the monitoringimage (on the CCD) are (x1, y1), the actual deviation (Δdy) from thelight axis on the CCD is expressed by an equation as follows.

[0203] [Equation 2]

Δdy=ccd _(—) p*(y1−239)

[0204] The fact that the focal distance of the camera unit 2 is ‘1’ andthe expression with Δdy of a moving amount of the grid line is expressedin an angle as follows.

[0205] [Equation 3]

φ′=tan−1(Δdy/1)

[0206] Furthermore, the distance g (x, y) up to the deviated grid linecan be calculated by using the relation between the angle from the lightsource element 11 of the grid line which has not moved yet and alreadystored into the data structure (Lh[n], angle) and the angle θ0 which alens plane 23 of the camera forms with the light source element 11. (Thefollowing equation 4 is a general equation of the triangulationtechniques.)

[0207] [Equation 4]

g(x, y)=L*tan θ*tan φ/(tan θ+tan φ)

[0208] where, φ=φ0+φ′

[0209] With the use of the equation 4, the whole of the distance g(x, y)corresponding to each moved grid line are calculated. For ensuringsafety, in calculable coordinates (crossing points of grid lines) inboth directions of X-axis and Y-axis, a smaller distance is used for thecalculation.

[0210] According to the present embodiment, the camera unit 2 is placedat the position at slanting 45° with respect to the light source unit 1,so that whole of distance z (x, y) can be calculated with one timeimaging by using the both parallax of the X-axis and the Y-axis. That isone of merits in using a laser pattern shaped like a grid.

[0211] The contents of processing regarding to the foregoing distancemeasurement by the image processing measuring part 33 are shown inflowcharts in FIG. 26-FIG. 28.

[0212] In the processing of distance calculation shown in the flowchartin FIG. 26, at first, initializing is performed (for every x and y, g(x, y)=0, m=0, n=0 j=0 and k=0) (step 2601).

[0213] Monitoring images encoded in binary form are stored into theframe memory FM1 (step 2602).

[0214] The reference image between the stored monitoring image and thereference image is extracted to store into the frame memory FM1.Accordingly, values of parts of moved grid lines become +values, valuesof parts of original grid lines become minus values and values ofunchanged parts become plus or minus 0 values (step 2603).

[0215] A point at FM (xm, ym)<0 (point ‘a’ in FIG. 24A) is retrievedalong the coordinates of gird lines (xm=Lh[n].xf[j], ym=Lh[n].yf[j])stored into the data structure Lh[n] of the horizontal grid lines (step2604). When the point ‘a’ is searched, processing is moved to a routinefor distance measurement (step 2701-step 2708, in FIG. 27). The searchprocessed in the step 2604 is performed to a whole of horizontal gridline (No, in step 2604, No, in step 2605 and step 2610). Where, ‘j’ is 0to Lh[n].max-Lh[n].min−1.

[0216] It is checked whether whole of search for horizontal grid linesis completed or not. When the search is completed, next, a search forvertical grid lines is performed (step 2607-2609).

[0217] A point at FM (xm, ym)<0 is searched along the coordinates ofgird lines (xm=Lv[n].xf[k], ym=Lv[n].yf[k]) stored into the datastructure Lv[m] of the vertical grid line (step 2607). When the point‘a’ is searched, processing is moved to a routine for distancemeasurement (step 2801-step 2808, in FIG. 28). The search processed inthe step 2607 is performed to whole of vertical grid line (No, in step2607, No, in step 2608 and step 2610). Where, ‘k’ is 0 toLv[m].max-Lv[m].min−1.

[0218] It is checked whether the search for vertical grid lines iscompleted or not (step 2609). This check is performed to whole ofvertical grid lines (NO, in step 2609, step 2613), when the search iscompleted the processing is completed contemporarily.

[0219] In a processing of distance calculation (a routine for distancemeasurement along with movements of grid lines in the horizontaldirection), at first, coordinates which have become

[0220] FM (fmx,fmy)<0 shown at point ‘a’ in FIG. 24A are stored as(fmx_s,fmy_s)(step 2701).

[0221] In a step 2702, a point whose coordinates FM (fmx, fmy) havebecome 0 again (point ‘b’ in FIG. 24A) is searched. The search isperformed up to the end of the grid line (NO, in step 2703, step 2704and NO, in step 2702).

[0222] In a step 2705, a middle point (fmx_s, fmy_s) (point ‘c’ in FIG.24A) of a section in which a position of a grid line is moved due to anincoming of an object is calculated from a present point (fmx, fmy).

[0223] In a step 2706, the grid line moved from the middle point (fmx_s,fmy_s) due to the incoming of the object is searched. The search isperformed for a point of FM1(x, y)>0 (a point ‘d’ in FIG. 24A) in aslanting 45° direction from the middle point.

[0224] In a step 2707, the search is performed in a right and leftdirections of the moved grid line to obtain the whole coordinates of thegrid line (coordinates on the frame memory FM1 shown as d to e to f at(a) in FIG. 24).

[0225] In a step 2808, g(x, y) of whole coordinates is calculated by theprinciple of triangulation techniques.

[0226] In distance processing (a routine for distance measurement alongwith movements of grid lines in the vertical direction), at first,coordinates which have become FM(fmx, fmy)<0 are stored as coordinates(fmx_s, fmy_s) (step 2801).

[0227] In a step 2802, a point that the coordinate FM1 (fmx, fmy) become0 again is searched. The search is performed up to the end of the gridline (NO, in step 2803, step 2804 and NO, in step 2802).

[0228] In a step 2805, a middle point (fmx_s, fmy_s) of a section inwhich a position of a grid line is moved due to the incoming of theobject is calculated from a present point (fmx, fmy).

[0229] In a step 2806, the grid line moved from the middle point due tothe incoming of the object is searched. The search is performed for apoint of FM1(x, y)>0 in a slanting 45° direction from the middle point.

[0230] In a step 2807, moved grid line is searched in the right and leftdirections and a whole of coordinates of the grid line is acquired.

[0231] In a step 2808, coordinates g (x, y) regarding to a whole ofcoordinate are calculated based on the principle of triangulationtechniques. Where, the same method for calculation as the foregoingmethod is applied to the vertical grid lines. But, as for points(crossing points of the vertical and horizontal grid lines) whosecoordinates g(x, y) have already calculated, if values which have justcalculated this time are greater than the values which have calculatedpreviously, the values calculated previously (smaller values) are givenpriority.

[0232] The decision part 35 checks whether the position of thecoordinates z (x, y) measured by the foregoing image processingmeasuring part 33 has entered into a specified area (a specified voxel)specified in the setting mode for types of outputs or not.

[0233] If the coordinates z(x, y) exist in the voxel which is set to‘stop’, the signal to stop the equipment is outputted immediately.

[0234] When the calculated coordinates z(x, y) is not in the voxel whichis set to ‘stop’ but in the voxel which is set to ‘warning’, a warningoutput is outputted. In this processing, the voxel, which is set to ‘noreaction’, is not checked.

[0235] Next, the following is an explanation of an incoming predictionfunction by means of the decision part 35.

[0236] The decision part 35, in the monitoring mode, collects points(point coordinates of surfaces of the incoming object) with coordinatessatisfying the following equation: z(x, y)<L (distance data L) andcalculates its gravity coordinates gt(xt, yt, zt) to store, then,acquires respective moving directions and moving speeds in the X, Y andZ-axis of the incoming object. Calculation equations of moving speed vare as follows.

[0237] [Equation 5]

vx=δgt/δx=(x[t−1]−xt)/measuring interval

vy=δgt/δy=(y[t−1]−yxt)/measuring interval

vz=δgt/δz=(z[t−1]−zt)/measuring interval

[0238] The decision part 35 calculates a predictive arrival position(coordinates) g [t+1] in the next measurement from the calculated movingspeed v and the present gravity position gt. As shown in FIG. 29A, forexample, when the incoming object moved in the monitoring space S andthe calculated gravity position gt changes as ‘g [t−1] to gt’, theincoming object M will be predicted to move to the back area S251 as faras it will go which is set to ‘stop’. In this situation, for example, itis considered that it is impossible for a case to secure safety of anoperator even stopping the equipment at the next measurement timing. Insuch case, the decision part 35 calculates the predictive arrivalposition g [t+1], and if the position g [t+1] exists in the area set to‘stop’, outputs the signal to stop the equipment at the point in time.

[0239] If the gravity position gt moves away toward the z-axis in themonitoring space S, or does not move (pass) toward the z-axis, thedecision part 35 considers that a dangerous factor is so low to clear awarning situation or keep the present situation.

[0240] If there is a plurality of incoming objects M simultaneously inthe monitoring space S, it is preferable for the decision part 35 tosplit the monitoring space S into applicable state of voxels andindividually manage at every voxel the gravity coordinates ofcoordinates z (x, y) belonging to each voxel, and calculate the movingdirections and moving speeds to predict a risk.

[0241] The decision part 35 uses the gravity positions, it is acceptableto use a point with the smallest value (the closest to the camera) forthe decision.

[0242] A flowchart of contents of processing regarding to the foregoingcalculation of the gravity coordinates and a prediction of arrivalpositions is presented in FIG. 30.

[0243] In a step 3001, distance measurment (calculation of coordinates z(x, y)) to the incoming object by the method for distance measurementshown in FIG. 25.

[0244] In a step 3002, coordinates of points satisfying an equation z(x,y)<L are added the calculated distance coordinates z (x, y) and thecoordinates z (x, y) are divided by the total number of the pointssatisfying the equation, so that gravity coordinates gt (x, y, z) at thetime (time t) are calculated.

[0245] In a step 3003 a voxel to which the gravity coordinates gt (x, y,z) belong (be positioned) is searched. (Octagon coordinates of a voxelare already known by calculation, so that the voxle to which the gravitycoordinates gt (x, y, z) belong can be specified in comparison with theoctagon coordinates.)

[0246] In a step 3004, if a voxel to which gravity coordinates g[t−1]belong have been set to ‘warning’ or ‘no reaction’ and also theappropriate voxel has been set to ‘stop’, it is decided that theincoming object has entered into a predetermined area (voxel), then, thesignal to stop the equipment is outputted in a step 3011.

[0247] In a step 3005, if the voxel to which the coordinatesg[t−1]belong have been set to ‘no reaction’ and also the appropriatevoxel with the coordinates gt has been set to ‘warning’ in a step 3006.

[0248] In a step 3007, the moving speed V of gravity is calculates asfollowing equations.

[0249] [Equation 6]

vx=(gxt−gx [t−1])/dt

vy=(gyt−gy [t−1])/dt

vz=(gzt−gz [t−1])/dt

[0250] In a step 3008, it is calculated that how far gravity moves bythe next sampling time (t+1) according to the following equations.

[0251] [Equation 7]

gx[t+1]=gxt+vx*dt

gy[t+1]=gyt+vy*dt

gz[t+1]=gzt+vz*dt

[0252] In a step 3009, it is searched that in which voxel gravitycoordinates g [t+1](x, y, z) will enter by the next sampling time (t+1).The search is performed in comparison with the octagon coordinates ofthe voxel.

[0253] In a step 3010, in a predictive position of the gravity at thetime (t+1) is in a voxel that has been set to ‘stop’, it is decided thatbeing in a dangerous situation, the signal to stop the equipment isoutputted in a step 3011. In the present embodiment, once the signal isoutputted, the equipment is stopped to operate until the equipment isreset (until a signal to restart an operation of the equipment isoutputted) through a user's operation.

[0254] In a step 3012, it is decided whether the voxel to which thegravity coordinates g[t−1] belongs have been set to ‘warning’ or not. Ifthe voxel has not been set to ‘warning’, processing will return to thestep 3001.

[0255] In a step 3013, Distances to a voxel which has been set to ‘stop’and which is respectively closest to the gravity positions gt[t−1] andg[t] are calculated to be defined respectively as d_g[t−1] and d_g[t].

[0256] In a step 3014, if it is recognized that a warning is nowoutputted from the decision part 35 and also that d_g[t−1]<d_g[t], it isdecided that the incoming object is moving away from the voxel which hasbeen set to ‘stop’ or that the object only has grazed the voxel whichhas been set to ‘warning’ so as to release the warning output.

[0257] Another method for self-diagnosis possible to be adapted to thethree-dimensional monitoring apparatus of the present invention isdescribed’as follows. In the embodiment, if the swing angle of the MEMS13 is 0° during the self-diagnosis, it is impossible for theself-diagnosis to check whether ON/OFF of the light source element 11 isperformed normally or not. Thus, in another method, the light sourceunit 1 and the camera unit 2 are used in another embodiment shown inFIG. 31. Only points different from the previous embodiment areexplained as follows.

[0258] A beam splitter 16 to irradiate a part of irradiating light fromthe light source element 11 to an optical fiber 17 is added to the lightsource unit 1. Other parts, except as described above, of the lightsource unit 1 are the same those of the previous embodiment.

[0259] A half miller 24 and an MEMS 23 to focus a laser beam that is abranch of the projecting light from the light source element 11 on a CCD25, are added to the camera unit 2. The branched laser beam iscontrolled to be focused at the both of the top end and the bottom endon the CCD 25 and not to be an obstacle of imaging of the pattern lightirradiated from the light source unit 1.

[0260] The camera unit control part 32 controls the MEMS 23 to scan thebranched laser beam from the right end to the left end on the CCD duringopening of the shutter.

[0261] Fields are split to even fields and odd fields, so that thecamera unit 2 forms an image on the top end over the CCD 25 when imagingthe even fields, and forms the image on the bottom end on the CCD 25when imaging the odd fields.

[0262] A timing chart of relations of control signals between the MEMS 1and the MEMS 2 is presented in FIG. 32. FIG. 32A shows a control signalassociated with swing angles θwx and θwy in the directions of the X-axisand the Y-axis of the MEMS 13, FIG. 32B shows a control signalassociated with an oscillation control signal (ON/OFF control) of thelight source element 11, FIG. 32C shows a control signal associated witha swing angle θvx in the direction of the X-axis of the MEMS 23 and FIG.32D shows a control signal associated with a swing angle θvy in thedirection of the Y-axis of the MEMS 23.

[0263] As shown in FIG. 32, during scanning of the MEMS 13 to thepattern light of one cycle of a character 8 in the monitoring space S,the MEMS 23 scans the branched laser beam over the CCD.

[0264] The self-diagnosis part performs the self-diagnosis with thefollowing steps.

[0265] 1) In the even fields, confirms that lines C1-C4 corresponding tothe ON/OFF control of the light source element 11 at the top end of animage shown in FIG. 33A.

[0266] 2) In the odd fields, confirms that lines C5-C8 corresponding tothe ON/OFF control of the light source element 11 at the bottom end ofthe image.

[0267] 3) If the lines C1-C8 are confirmed in the steps 1) and 2), it isconsidered that the self-diagnosis is completed without problems.

[0268] In the method for self-diagnosis described above, even though theswing angle of the MEMS 13 is 0°, the self-diagnosis can be performedaccurately.

[0269] Although the screen 5 is used as a projection plane for thepattern projection in the foregoing embodiment, a wall and the likeplaced behind the monitoring space S are useful.

[0270] As described above, according to the present invention, athree-dimensional monitoring apparatus capable of detecting an incomingof an object into a predetermined three-dimensional space to bemonitored can be provided with high accuracy.

What is claimed is:
 1. A three-dimensional monitoring apparatus,comprising: an irradiating means for irradiating predetermined patternlight to a three-dimensional space to be monitored; an imaging means forimaging a projection pattern projected by irradiating the pattern lighton a surface of an incoming object existing in the space to be monitoredand on a surface of a predetermined body composing of a background ofthe space to be monitored to capture image data; a position measuringmeans for acquiring position information on the incoming object into thespace to be monitored based on the comparison between the image datacaptured via the imaging means when there is the incoming object intothe space to be monitored and reference image data corresponding to theimage data captured via the imaging means when there is no incomingobject into the space to be monitored; and a decision-output means foroutputting a device control signal based on the position informationcalculated by the position measuring means; wherein the irradiatingmeans has a scanning mechanism capable of scanning the pattern light ina predetermined area by controlling a direction of irradiation of thepattern light; and wherein the imaging means captures image datacorresponding to a combined projection of a plurality of instantprojection patterns irradiated to the space to be monitored at apredetermined direction and timing through a scanning of the patternlight in the space to be monitored by the scanning mechanism.
 2. Athree-dimensional monitoring apparatus as claimed in claim 1, whereinthe scanning mechanism is an MEMS (micro electro mechanical system)comprising a light reflector and a support member controlled by atorsion rotation through electric induction to support the lightreflector turnably.
 3. A three-dimensional monitoring apparatus asclaimed in claim 1, wherein the three-dimensional monitoring apparatuscomprises an irradiating device as the irradiating means and an imagingdevice as the imaging means; and wherein the position measuring meanscalculates three-dimensional coordinates for the incoming object intothe space based on the principle of triangulation techniques by usingknown shapes or patterns of the pattern light and displacements acquiredfrom image data captured via the imaging device when there is theincoming object in the space to be monitored and the reference imagedata captured in advance.
 4. A three-dimensional monitoring apparatus,comprising: an irradiating means for irradiating predetermined patternlight to a three-dimensional space to be monitored; an imaging means forimaging a projection pattern projected by irradiating the pattern lighton a surface of an incoming object existing in the space to be monitoredand on a surface of a predetermined body composing of a background ofthe space to be monitored to capture image data; a position measuringmeans for acquiring position information on the incoming object into thespace to be monitored based on the comparison between the image datacaptured via the imaging means when there is the incoming object intothe space to be monitored and reference image data corresponding to theimage data captured via the imaging means when there is no incomingobject into the space to be monitored; a decision-output means foroutputting a device control signal based on the position informationcalculated by the position measuring means; and a means for specifyingarea selected freely as a specified area through a user's operation fromthe space to be monitored divided into a plurality of areas virtually inadvance and for performing a setting of types of outputs selected freelythrough the user's operation from a plurality of types of operationsprepared in advance by specified area; wherein the position measuringmeans acquires specified information on the area with the presence ofthe incoming object as the position information on the incoming object;and wherein the decision-output means outputs device control signalbased on the types of outputs set to a specified area via the positionmeasuring means.
 5. A three-dimensional monitoring apparatus as claimedin claim 4, wherein the setting of the types of outputs is performed byvoxel (volume pixel) partitioned based on a shape or a pattern of thepattern light.
 6. A three-dimensional monitoring apparatus as claimed inclaim 4, wherein the specified area images in a state that the incomingobject is positioned at a desired position in the space to be monitoredand automatically specified by extracting an area where the incomingobject exists.
 7. A three-dimensional monitoring apparatus, comprising:an irradiating means for irradiating predetermined pattern light to athree-dimensional space to be monitored; an imaging means for imaging aprojection pattern projected by irradiating the pattern light on anincoming object existing in the space to be monitored and on a surfaceof a predetermined body composing of a background of the space to bemonitored to capture image data; a position measuring means foracquiring position information on the incoming object into the space tobe monitored based on the comparison between the image data captured viathe imaging means when there is the incoming object into the space to bemonitored and reference image data corresponding to the image datacaptured via the imaging means when there is no incoming object into thespace to be monitored; a decision-output means for outputting a devicecontrol signal based on the position information calculated by theposition measuring means; and a means for specifying area selectedfreely as a specified area through a user's operation from the space tobe monitored divided into a plurality of areas virtually in advance andfor performing a setting of types of outputs selected freely through theuser's operation from a plurality of types of operations prepared inadvance by specified area; wherein the position measuring means measuresa moving state of the incoming object based on time-series changes of aplurality of monitoring image data acquired sequentially; and whereinthe decision-output means specifies predictive arrival area of theincoming objects based on a measurement result by the position measuringmeans and outputs a device control signal based on the predictivearrival area and the types of outputs determined in the predictive area.8. A three-dimensional monitoring apparatus, comprising: an irradiatingmeans for irradiating predetermined pattern light to a three-dimensionalspace to be monitored; an imaging means for imaging a projection patternprojected by irradiating the pattern light on a surface of an incomingobject existing in the space to be monitored and on a surface of apredetermined body composing of a background of the space to bemonitored to capture image data; a position measuring means foracquiring position information on the incoming object into the space tobe monitored based on the comparison between the image data captured viathe imaging means when there is the incoming object into the space to bemonitored and reference image data corresponding to the image datacaptured via the imaging means when there is no incoming object into thespace to be monitored; a decision-output means for outputting a devicecontrol signal based on the position information calculated by theposition measuring; and a self-diagnosis means for confirmingappropriately whether the projection pattern based on the pattern lightirradiated via the irradiating means is matched with a predictiveprojection pattern and for ensuring normal operations of eachconfiguration means when the matching is confirmed.
 9. Athree-dimensional monitoring apparatus as claimed in claim 8, whereinthe irradiating means has a scanning mechanism capable of scanning thepattern light within a predetermined area by controlling the directionof irradiating of the pattern light; and wherein the imaging meanscaptures image data corresponding to a combined projection of aplurality of instant projection patterns irradiated to the space to bemonitored at a predetermined direction and timing through a scanning ofthe pattern light in the space to be monitored by the scanningmechanism.
 10. A three-dimensional monitoring apparatus as claimed inclaim 8, wherein the three-dimensional monitoring apparatus has aself-diagnosis means for detecting automatically the presence or absenceof unusual occurrences of operations of the monitoring apparatus itselfbased on whether a predictive checked pattern is appeared normally inthe image data captured by irradiating directly a part of the lightirradiated from the irradiating means on an imaging plane of the imagingmeans.
 11. A three-dimensional monitoring apparatus as claimed in claim10, wherein the three-dimensional monitoring apparatus has a laser beamsource as the irradiating means and defines the checked pattern byflashing timing and scanning timing of the laser beam source.
 12. Athree-dimensional monitoring apparatus, comprising; an irradiating meansfor irradiating predetermined pattern light to a three-dimensional spaceto be monitored; an imaging means for imaging a projection patternprojected by irradiating the pattern light on a surface of an incomingobject existing in the space to be monitored and on a surface of apredetermined body composing of a background of the space to bemonitored to capture image data; a position measuring means foracquiring position information on the incoming object into the space tobe monitored based on the comparison between the image data captured viathe imaging means when there is the incoming object into the space to bemonitored and reference image data corresponding to the image datacaptured via the imaging means when there is no incoming object into thespace to be monitored; a decision-output means for outputting a devicecontrol signal based on the position information calculated by theposition measuring means; wherein the imaging means and the irradiatingmeans are positioned and placed in a manner that an angle which avirtual line combining the imaging means and the irradiating means formswith a horizontal plane becomes approximately 45°, so that positioninformation on the incoming object is measured by a parallax.
 13. Athree-dimensional monitoring method for monitoring simultaneously athree-dimensional space to be monitored from a plurality of directionsby using a plurality of three-dimensional monitoring apparatuses asclaimed in claim
 1. 14. A three-dimensional monitoring system includinga three-dimensional monitoring apparatus and equipment to be controlled;wherein the monitoring apparatus comprises: an irradiating means forirradiating a predetermined pattern light to a three-dimensional spaceto be monitored; an imaging means for imaging a projection patternprojected by irradiating the pattern light on a surface of an incomingobject existing in the space to be monitored and on a surface of apredetermined body composing of a background of the space to bemonitored to capture image data; a position measuring means forcalculating position information on the incoming object into the spaceto be monitored based on the comparison between the image data capturedvia the imaging means when there is the incoming object into the spaceto be monitored and reference image data corresponding to the image datacaptured via the imaging means when there is no incoming object into thespace to be monitored; and an output means for outputting a controlsignal to the equipment based on the calculated position information.